DOCUMENT RESUME 



ED 262 478 



EA 018 034 



AUTHOR 
TITLE 



PUB DATE 
NOTE 



PUB TYPE 

EDRS PRICE 
DESCRIPTORS 



IDENTIFIERS 



Hamnond, Peirce 

What Do You Mean Objective and Collaborativfi? or, How 
One District Tries to Keep Its Nose Clean and Its 
Hands Dirty at the Same Time. 
1 Apr 85 

14p.; Paper presented at the Annual Meeting of the 
American Educational Research Association (Chicago, 
XL, March 31-April 4, 1985). 

Speeches/Conference Papers (150) — Viewpoints (120) 
KFOl/PCOl Plus Postage. 

^Compliance (Legal); Cooperation; *Court Liiigation; 
Data Collection; ^Desegregation Litigation; 
Educational Administration; Educational Planning; 
^Educational Testing; Elementary Secondary Education; 
♦Evaluation Methods; Federal Regulation; Legal 
Responsibility; ^School Desegregation; Test 
Validit/ 

♦Cleveland Public Schools OH 



ABSTRACT 

The problems existing during the reorganization of 
the Cleveland School District into a' unitary administration are 
presented from the perspective of the head of the Research and 
Analysis Department. The district had been fighting a desegregation 
case for 9 years and operating in a receivership with a triple 
bureaucracy imposed upon it (the normal school dictrict bureaucracy, 
court-mandated reporting procedures, and state-imposed procedures). 
-As a result many problems still exist. Testing is a major problem, 
because the triple* requirements have resulted in overtesting. The 
district also faces problems evaluating programs. To better 
understand these an<? other problems, the district is developing 
better relationships between schools and the central administration 
and is working in a manner similar to regional exchanges (problem 
clarification to resource identification, to resource linkage). The 
Research and Analysis Department is developing policies based on 
court dociURents and investigating and analyzing the impact of 
policies after implementation. The district recognizes that it has 
been oriented toward reacting to immediate prescribed tasks due to 
these problems. It is now working to orient itself to the end of the 
desegregation case by combining the Final Standards with prescribed 
actions, with permanency, and with evidence that the district is 
functioning beyond the prescribed requirements. (MD) 
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In August, 1982, Cleveland's hard-fought' desegregation case was nine 
years old. The district was in receivership. The Federal District Court had 
come down hard on the district due to the recalcitrant behavior of the board 
and members of the administration. Some improvements were at hand, however, 
1-irst, a two year period of dual administration, "Deseg" at one end of the 
third floor, the "Regular" Superintendent at the other, was ending. Second, a 
new superintendent, Cleveland's first black superintendent, Frederick 
HoUiday, was to take offixie in three months. 

The reorganization which was necessary to move the district to a 
unitary administration included six cabinet-level departments under the 
superintendent and deputy superintendent, three reporting to each. One of 
these was the Department of Research and Analysis, reporting directly to the 
superintendent. In June, 1983, I assumed the position of "Chief" of that 
department . 

The department was a recombination of Deseg and Regular functions. 
Its personnel had served on each side of the house. Jt included four 
divisions each headed by a director: Testing and Evaluation was the largest 
unit. It was, and is, directed by a strong, experienced evaluation and 
testing professional. It was the heart of the office in many ways. Its 
leader was the most respected of the old staff still in Research. Further, 
its functions, along with proposal writing, were the classic Research 
functions and served to provide some stability in what had been a period of 
seemingly constant change lasting for several years. During those years, the 
fortunes of Research and of several of its key figures had gone up and down 
and up and down again, until no one was really certain what had improved, what 
worsened, who was in charge and for how long, or what the rules were. 
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The changes Included the addition of three other divisions. 
Research Dissemination and Proposal Development also performed functions which 
members of the research staff were familiar and comfortable with. 
Dissemination was largely a matter of doing ERIC and other literature 
"searches". Proposal development consisted of technical assistance, including 
technical writing, for proposals which either were relatively standard (e.g.. 
Title I, now Chapter One, and its state counterpart) or which originated 
elsewhere. 

Policy, Planning and Analysis was a new division. The major clue to 
its function came from its name. No one was certain if the focus of the 
division should be to consider planning and analysis in relation to policy 
development only; to consider the three tasks, policy development, planning, 
and analysis separate although possibly related items; or to conc6i*trate on 
one or two of the assigned functions (e.g., policy, or planning, or analysis). 
The situation was not helped because there was a lack of direction from the 
Acting Superintendent who established the divisions (who became Deputy in 
November, 1982, then left the district in June, 1983), the hiring of a 
high-powered outside consultant firm to write policies, and the appointment of 
a division director who had little experience in the area who simultaneously 
was made Acting Chief of the Department. (For that matter, none of the 
division staff had much policy development or long-ranged planning 
expertise— although good analytic skills were present.) Policy development 
was, however, the main task the division had focused on by my arrival two 
years ago. 

The fourth division, also new, was Desegregation Monitoring and 
Special Studies. Here, too, there were no explicit guidelines. The division 
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borrowed its approach from evaluation through observation coupled with some 
data analysis and from the work of the Court-mandated Office on School 
Monitoring and Community Relations, The latter has served as the Courtis eyes 
and ears and, in some respects, its surrogate* since its inception in 1978 • 

The department was sound in its staffing thanks largely to tha 
efforts of Margaret Fleming who had been the leader of its predecessor 
organizations. Dr. Fleming had, in 1978, been made Deputy Superintendent. In 
1983, she, along with the other ranking administrators at the cabinet levels 
from both ends of the hall, was made a Special Assistant to the 
Superintendent** . 

We were operating in what amounted to be a triple bureaucracy in 
1983. In addition to the normal urban school district bureaucracy was imposed 
that of Court-mandated reports, studies, and mandates and this was topped off 
by State-imposed procedures due to the system's indebtedness to the State. 
Both "extra" bureaucracies placed some helpful strictures, priorities, 
necessities, and goals on the district. But both were resented and resisted. 

Resentment, more than resistance, was a prevailing mood in Research 
and Analysis in 1983. This had four principal causes. First, many in the 
department had been in more than one job in more than one division of the 
district during the previous five years. Second, many had devoted much effort 
to attempting to help the district through this difficult period and had seen 



*Whlle being explicit that it does not speak for the Court, O.S.M.C.R. does 
have Court-given authority to comment on a wide range of district-produced 
evidence, including various required periodic reports, as well as to carry 
out special studies of its own. 

**Both administrative heads prior to the re-unified administration, the 
Superintendent and the Administrator for Desegregation, left the employ of 
the Cleveland City School District in 1982. 
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some significant part of their effort go for naught or be trampled underfoot, 
resulting in cynicism, bum-out, or resignation. Third, Margaret Fleming, 
respected by all, admired by many, had taicen a bad beating after having been 
elevated to Deputy Superintendent— many saw her as a scapegoat whoee 
professional standards were such that she was loyal to a fault. Fourth, 
whereas some two years earlier the administrative staff in the department were 
all told to get certificates in order, just months before my arrival the 
Acting Superintendent had "demoted" the department to a classified 
(non-certificated) entity, complete with lower salary levels and the stigma cf 
association not with Curriculum and Instruction but with the Business 
Department, trades workers, and secretaries. 

Whatever may have been expected of me when I arrived, I was 
well-treated. Margaret Fleming has been, as always, the consumate 
professional. My predecessor was completely helpful. He left the district 
some three months after my arrival, but was entirely cooperative until then. 
I felt little resentment from the staff and some welcome. Most were quite 
willing to give me a chance. 

The problems confronting us then are mostly still there. 
In Testing there were two major problems with a third on the 
horizon. First, we were overtesting. We still are, but we have reached, in 
principal, an understanding with the Department of Curriculum and Instruction 
regarding cutting back. We test as follows: CTBS Reading in grades 1-12; 
Cleveland Reading Competency Test in grades 1-9; CTBS Language in grades 3-6, 
8, and II; CTBS Math in grades 3-8; CTBS Science in grades 5 and 8; Stanford 
Diagnostic Reading Test in grades 2, 4, 5, 7, 8, and 9; Iowa Test of Basic 
Skills Social Studies in grades 5, 6, 7, and 9; Ohio Vocational Achievement 
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Test In vocational schools, grades 10-12; Cognitive Abilities Test In grades 
2, 3, 6, and 8; and the Ohio Interest Survey In grades 8 and 10. This year we 
made a break-through In making the Differential Aptitudes Test optional In 
grade 8. With it Included, an eighth grader wae subjected to ten different 
tests, extsndlng over 855 minutes, i.e., over fourteen hours of instruction 
lost. Aside from eighth grade, the lowest amount of time spent testing was 
205 minutes on two tests in grade 1; the highest was 580 minutes for five 
tests in grade 11 or 555 minutes for seven tests in grade 5. Every grade but 
first had at least 390 minutes of testing. 

In addition, the State has mandated a rather modest form of 
Competency-Based Education in Reading, Mathematics, and English Composition 
phased-in through the 1989-90 school year. This brings us to the second major 
problem. The Acting Superintendent brought in an outside consultant to 
develop a reading competency test for the district. The "Deseg" side of the 
house had seen competency testing as a means of focussing attention very 
clearly on reading Instruction. This it had done with much fanfare. Indeed, 
the district was committed well beyond the state competency testing mandate 
which required testing at least once in j^rades 1-4, at least once in 5-8, and 
at least once in 9-12 and which imposed no sanctions. The district was 
committed to annual testing in all grades 1-9. In addition, failure to pass 
the test meant failure to pass the grade except in grade nine. In order to 
receive a diploma, one had to pass the ninth grade test. This commitment was 
loud and public. Changes might be seen as expecting less and, therefore, 
discriminatory. 

A pilot test and then a field test of the competency test pool of 
items had been conducted in Spring, 1983. Logistical problems, particularly 
regarding the former, had created an uproar. The objectives upon which the 

7 

ERIC 



6 



test had been based had been Introduced part way through the 1982-83 school 
year with little staff development and confusing directions. Results were 
announced based on an assumption that Cleveland had very high standards both 
with respect to the objectives which had been set and as regards passing 
scores. In order to pass, It was judged, one had to receive credit for 
mastering at least three quarters of the objectives. One received credit for 
mastering an objective If one correctly answered at least three out of the 
four Items testing that objective correctly. 

Another uproar. In grades one and two, some forty to fifty percent 
of the students would have passed under these rules. But In grade three, only 
about sixteen percent and the results were worse for the upper grades, with 
less than one percent showing this level of mastery In ninth grade. 

The new Assistant Superintendent for Curriculum and Instruction and 
I recognized that we had Inherited a mess. Politically, the Board and 
Superintendent were conmiltted to higher standards through a tough competency 
test. But 99Z failure?! We called In two panels of experts, Reading experts* 
to examine reading procedures and objectives and measurement experts** to 
advise on testing and measurement procedures. As a result. Curriculum and 
Instruction committed to a re-examlnatlon of the reading objectives and we 
followed with the commitment tc redo the test to follow the revised objectives 
and to develop an empirically-based cut-score. We also managed to get the 
district to back off of falling students at every grade level based solely on 
the test. It was to be one factor In the teacher *s decisions. The diploma 



*Jeanne Chall, Carl Smith, and Dorothy Strickland 
**Ronald Hambleton, Paul LeMahleu, and Jason Mlllman 
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sanction remained, but not to be effective until 1992 when the first seniors 
to have attended only desegregated schools In Cleveland were to graduate. 

We still face many problems In this area. Cut scores, summer school 
testing, development of tests In mathematics and composition, and the 
relationship between the competency test and the standardized norm-referenced 
reading test used" by the district are some of them. They are compounded by 
changing political circumstances; a new, possibly Interim, superintendent with 
different, more skeptical, views of competency testing than his predecessor; 
and more vocal questioning in the form of a written report by the Office of 
School Monitoring and Community Relations. 

. The testing problem on the horizon stems from a Court Order which 
requires that we safeguard to be certain not only that "All tests, whether 
standardized, criterion-referenced, or teacher-made, be developed, 
administered, and scored in a nondiscriminatory manner" but that, in addition, 
"the results of such tests be used in a non-discriminatory manner*" [emphasis 
added]. How to assure the Court that this is so is a. problem. 

Our evaluation shop faces the problem of too much to do. We have 
moved beyond the project evaluations of Chapter I on which most of us cut our 
teeth. We must still do such evaluations. But we have larger issues to face. 
We must now evaluate whole programs irrespective of funding: the district's 
Affirmative Reading Program; its Technical-Vocational Education Program; its 
Magnet Schools Program; and others. In addition, our current board and 
superintendent constantly ask evaluative questions. They want studies of 
everything based upon "Indicators of Effectiveness" for everything. We are 

*Flnal St andards for Implementation of the Remedial Orders , United States 
District Court, April 24, 1981. 
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exploring several approaches. First, ouch of what is desired is policy-based 
and so our Policy, Planning and Analysis staff is also involved. Second, each 
department, division, and school must develop an Operational Plan each year 
which is to Include measurable objectives. These could form the basis for 
answering many of the questions. Third, these same objectives could be 
translated intp simple, easily collected, easily understood Indicators of 
Effectiveness on which Department or Division heads as well as Research and 
Analysis staff members could report. 

The main issue in dissemination has been to get our hands more dirty 
in ways which contribute more fully to better-operating schools. ERIC 
searches are fine, but we are now attempting to develop working relationships 
with schools which will allow us to better understand their problems and ways 
which we can assist in solving or managing them and better help them to know 
what we and research in general can do to be of service to them. This 
attitude was one I developed at NIE. One of the surprises awaiting me in the 
"real world" of an urban district was that this attitude did not exist and was 
resisted out there. 

Despite such inertia, I have been determined to press on into this 
area, seeing it as the key to our future operation as a contributor to 
improved educatio.: in Cleveland. In searching for models, we have been 
inspired by the work of the R & D Interpretation Service supported by NIE and 
currently operated by the Appalachia Educational Laboratory under David 
Holdzkom. This model has helped us to think about our role as research 
digesters, interpreters, synthesizers, and packagers as well as that of 
advocates for this use of knowledge. 

RDIS was also instructive to us in our need to be sensitive to the 
desires, needs, and capabilities of those who we wish to serve. This attitude 
was also present in work done in Fairfax County, Virginia under Todd Endo. 
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Todd shared information about these efforts with us and encouraged us to get 
out into school sites. This effort began this fall, using a questionnaire 
asking: 

. If you could work on one problem area that exists within 
your school, what would that problem area be? 

. What factors do you think contribute to/cause this problem? 

. What resources do you think you would need to help solve 
this problem? 

• How would you see someone from the Research Department 
helping you solve this problem? 

The visits proved to be profitable. Direct communication between 
researcher and site administrator had seldom taken place in the past. The 
researchers were able to convey knowledge about the general functions 
performed by the Research Department as well as about specific services that 
could be utilized by school site personnel. The school site administrator 
found a r'^Rource within the school district that was interested in 
understanding the problems that impeded efforts to Improve the learning 
process within their school. 

After the visits were completed, the Research Dissemination staff 
reviewed all interview data. As expected, a multitude of problems surfaced as 
a result of these interviews. Surprisingly, the most frequently mentioned 
concern pertained to student behavxor in the elementary school lunchroom. 
During the two hours devoted to lunch (approximately one-third of a typical 
school day), the inappropriate behavior exhibited by students encouraged 
disruption within the classroom. This focus has provided us with a small 
beginning, but a practical, workable concern has been identified. 

We 3.re now working in a manner similar to that often used by 
Regional Exchanges: problem clarification to resource identification (usually 
within the system) to resource "linkage", technical assistance, training, 
T:rnn' interpretation, further study and so on. 

ERIC 
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Our efforts in policy work have now received much needed 
orientation. The Board, with some prompting from us and fron O.S.M.C.R. , has 
become concerned with policy as (1) a vehicle for demonstrating to the Court 
its connnitment to institutionalizing desegregation practices and (2) a means 
of expressing its will to the administration and providing a means for holding 
the administration accountable to its expressed views. These changes have 
served to focus the division of Policy, Planning and Analysis on assisting in 
deriving needed policies based on Court documents supplemented by data, 
knowledge, and experience and on investigating and analyzing the impact of 
policies on behavior. The latter comes through quarterly review sessions with 
those most impacted by policies from central office, school sites 
(principals), and the community. Planning remains something of an 
abstraction, but its day will come as these other functions mature. 

Desegregation Monitoring and Special Studies has also undergone 
changes and also contemplates the possibility of more in the future. Its 
orientation was changed, shortly after my arrival, from field monitoring 
combined with a rather diffuse pattern of examination of policies and of 
progress towards accomplishment of required actions. The required actions 
were from a diverse body of reports and comments from O.S.M.C.R. Under my 
direction, these were gathered into one Monthly Progress Report. The 
Desegregation Monitoring unit then focussed its attention on the district's 
progress in finishing these. This unit has also served to coordinate much of 
the activity in moving other departments into compliance. 

Over time it became clear that the district needed to accomplish 
more than this set of "fixed pieces". We needed to institutionalize our 
changed behavior to provide the Court and the community some basis for 
confidence in restoring to us the public trust. .We, further, needed to take 

12 
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some Initiative to show good faith in both corrective and proactive ways. 
That is, we needed to show that we could and would discover and fix our own 
errors and that we could and would be proactive and affirmative in forwarding 
the cause of desegregation on our own initiative. 

That we were too oriented to the immediate, prescribed tasks was 
clear for all to see, but made much easier to see and to take initiative to 
correct due to the fact that the Chief of Research and Analysis sits on the 
Superintendent's Cabinet. This has allowed me to begin a process to orient 
the district toward the end of the Desegregation case by combining the Final 
Standards, to which the judge will hold us and on which we will have to 
present our evidence that we are indeed in compliance, with both accomplished 
and to-be-accomplished required (prescribed) actions, with evidence of 
permanency or institutionalization (policies and their implementation) , and 
with evidence of our going beyond the prescribed. 

But here is where hands dirty (helping analyze and solve problems) 
and nose clean must be in balance. I cannot lose my perspective as an 
objective, if caring and motivated, observer. My division of Desegregation 
Monitoring must be valued, respected, believed, and above reproach. I also 
cannot do other people's work for them, nor allow Research people to be 
co-opted to do it. Yet I and we must be constructive. We must avoid being 
seen as imposing requirements in order that ownership and true 
institutionalization, true belief, true change may occur. 

This issue, credibility on the one hand, constructive contribution 
on the other, is a constant and persistent issue. The current superintendent 
wants us to "cross the line". He values our intelligence and our judgment and 
says he wants us to serve as his staff. We report to him. But our 
intellectual lineage is made of more conservative stuff. We believe that the 
odds need to be at least 19:1 against error before we commit. We want 
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replication even if those odds are quoted to us. Are we ready for the world 
of politics and handball. Are we ready for the next administration to ask us 
where we got off making all these pronouncements. As we ready to abandon the 
ivory tower of untouchability and unreality in quest of the real, "real 
world"? 
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